A METHOD AND APPARATUS FOR COLOR IMAGE PROCESSING, 
AND A COMPUTER PRODUCT 



FIELD OF THE INVENTION 
5 The present invention relates in general to a 

technology for color image processing. More particularly, 
this invention relates to the color image processing method 
for enabling removal of show-through when matter is printed 
on both the sides of a paper. Here, "show-through" means 
10 a phenomenon that the matter printed on one side of the paper 
is seen from the other side of the paper. 

BACKGROUND OF THE INVENTION 

Fig. 4 explains the phenomenon of show-through, and 

15 it is assumed that the object shown are colored ones. In 
accordance with wide spread of equipments which can capture 
or print color images , the use of color images in magazines , 
catalogues, advertisement, or newspapers have become common 
The examples of equipments which can capture color images 

20 are color scanners, digital cameras, and digital color 
copiers. In almost all the cases an image or matter is 
printed on both the sides of the paper. However, sometimes 
the image on one side of the paper can be seen even from 
the other side. This fact is shown in Fig. 4. The image 

25 4A is printed on the back side of the paper and the other 



images are printed on the front side of the paper. However, 
the image 4A printed on the back side of the paper is seen 
event from the front side. Further, in many cases, a color 
document has a different background color, or has image 
5 patterns, other patterns , and photographs together with one 
another. Therefore, the problem to remove "show-through" 
from the input image by performing digital image processing 
has been regarded as being difficult to handle although it 
is important for improvement in image quality. 

10 Methods for removing show-through, on special 

equipments such as a book scanner or a double-side scanner, 
are known. Such methods have been disclosed, for example, 
in Japanese Patent Nos . HEI 07-87295A, HEI 08-265563A, HEI 
09-205544A, HEI 09-233319A, or United States Patent Nos. 

15 5973792 and 5932137. 

The United States Patent No . 5932137 discloses amethod 
and apparatus. What is disclosed is a technology for 
mitigating the effects of show-through by scanning the both 
the sides of the paper and and storing the scanned images. 

20 The invention according to this publication is realized by 
scanning a duplexed document with show-through derived from 
the second side at least on the first side, storing a first 
side image and a second side image, generating a 
representation (mirror-image conversion, registration of 

25 images) corresponding to a component attributable to 



show-through from the second side image to the first side, 
and correcting (using a show- through coefficient) the image 
by removing the show- through from the firstsideasa function 
of the representation of the second side image. Further, 
5 the image processing method for mitigating the effects of 
show-through is performed by scanning a first side and then 
a second side of a duplexed document with show-through 
derived from the second side on the first side, storing the 
first side image and the second side image, generating a 

10 representation (mirror-image conversion, registration of 
images) corresponding to a component attributable to 
show-through from the second side image to the first side 
(and from the first side image to the second side) , and 
correcting (using a show-through coefficient) the image by 

15 removing the show-through as a function of the representation 
of the second side image (first side image) from the first 
side (second side). Further, in this publication, there 
has been described a document printing system which mitigates 
the effects of show-through for printing an image with 

20 mitigated effects of show-through . More specif ically , this 
system comprises a scanner that scans a duplexed document 
with show-through derived from the second side at least on 
the first side, and stores images on the first side and the 
second side, an image processing circuit (mirror-image 

25 conversion, registration of images) that generates a 



representation corresponding to a component attributable 
to show-through from the second side image to the first side, 
and an image processing unit (using a show-through 
coefficient) that removes the show-through from the first 
5 side as a function of the representation of the second side 
image, and corrects the image. 

In the above-mentioned methods, at first, the input 
images on both sides of the paper are accurately aligned 
in their position, and the images on both sides are compared 

10 to each other to estimate a transmission coefficient of paper . 
By subtracting a component attributable to show-through from 
the front side image using the transmission coefficient, 
the show- through is removed. However, there is a problem 
in these methods such that the methods can not be applied 

15 to any other equipment except specific input equipment that 
can input and store images on both sides and accurately 
register the images or such environments. 

As another methods for removing the show-through, 
there have been proposed methods based on analysis or 

20 binarization of image density using only information for 
a single-side image. For example, the methods have been 
proposed in Japanese Patent Nos. HEI 11-187266A, HEI 
11-41466A, [J. Sauvola, T. Seppanen, S. Haapakoski, andM. 
Pietikainen, "Adaptive document binarization," Proc . 4th 

25 Int. Conf. Document Analys i s and Recognition (Ulm, Germany) , 



August 18-20, 1997, pp. 142 - 146.], United States Patent 
Nos. 5646744 and 5832137. However, when the image is 
complicated, it is difficult to discriminate between a front 
side image and show-through using density or color 
5 distribution. There comes up risk such that a character 
having low contrast to' a background in particular, for 
example, a character in yellow on a white background may 
be processed as show-through. 

Thus, the method of scanning and storing the images 

10 has a problem that it can be only utilized on special 
eguipments and can not be applied to other equipments . The 
special equipments are the ones that can input and store 
images on both sides of the paper and can accurately align 
the positions of both the images. 

15 On the other hand, the method of using the image density 

has a problem that it is difficult to discriminate between 
a front side image and show-through using density or color 
distribution when the image is complicated. With this 
method, there may be a case that a character having low 

20 contrast to a background in particular, for example, a 
character in yellow on a white background is processed as 
show-through . 



5 



SUMMARY OF THE INVENTION 

It is an object of this invention to provide a color 
image processing method which is applied to general-purpose 
equipment without depending on a specific input device and 
5 which can remove show-through from a color image by using 
only information for a single-side image when a color 
document printed on both sides and having show-through 
derived from the second side on the firstsideistobe scanned . 
The object of this invention is also to provide a color image 

10 processing apparatus for executing the method, and a 
computer-readable recording medium where .a program for 
making a computer execute the method and the functions of 
the apparatus is receded. 

According to one aspect of this invention, a component 

15 corresponding to the show-through is removed based on 
detection of the intensity of an edge and color threshold 
processing for a digital color image obtained by digitally 
inputting a document color-printed on both sides of paper. 

According to another aspect of this invention, a 

20 component corresponding to the show-through is removedbased 
on detection of the intensity of an edge and color threshold 
processing for a digital color image obtained by digitally 
inputting a document color-printed on both sides of paper, 
an image from which the show-through has been removed 

25 obtained as a result of the processing is compared with edge 



distribution of an original image. In the show-through 
removed, image, the color threshold processing is again 
applied to the periphery of an edge which does not exist 
in the original image to correct the image. 
5 According to still another aspect of this invention, 

the color threshold processing is performed by setting a 
window of a predetermined size in a portion where no edge 
exists in the edge distribution obtained by binarizing the 
detected edge intensity, locally classifying pixels within 

10 the window into two colors, and replacing the color of the 
pixels within the window with the color with high brightness 
(light color) . 

According to still another aspect of this invention, 
a component corresponding to the show-through is removed 

15 based on detection of edge intensity of a front side image 
of a document and color threshold processing without having 
to input the rear side of the document. 

According to still another aspect of this invention, 
in the show-through removed image, the color threshold 

20 processing is again applied to the periphery of an edge which 
does not exist in the original image, and when the image 
is to be corrected, the size of the window according to the 
invention is set to a smaller size as compared to that in 
the previous processing. 

25 According to still another aspect of this invention. 



in comparison between the show-through removed image and 
the edge distribution of the original image, at each pixel, 
the edge intensity calculated in the original image is 
subtracted from the edge intensity calculated in the 
show- through removed image. As a result, the value higher 
than the predetermined threshold is determined as an edge 
which does not exist in the original image in the show-through 
removed image. 

Other objects and features of this invention will 
become apparent from the following description with 
reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of modules showing a color 
15 image processing apparatus according to an embodiment of 
this invention; 

Fig. 2 is a flow chart showing a color image processing 
method according to an embodiment of this invention; 

Fig. 3 is a flow chart showing in detail the color 
20 image processing method according to an embodiment of this 
invention ; 

Fig. 4 shows an example of a color image with 
show-through ; 

Fig. 5 shows an edge intensity image generated by a 
25 vector Sobel operator applied to the image in Fig. 4; 

8 



Fig. 6 shows a result of binarizing the color edge 
intensity in Fig. 5; 

Fig. 7 is a flow chart showing locally-executed color 
threshold processing using a window of size S in detail; 
5 Fig. 8 is a flow chart showing locally-executed color 

threshold processing using a window of size S in detail; 

Fig. 9A and Fig. 9B show examples of formation of runs 
in order to estimate a background color; 

Fig. lOA and Fig. lOB show images as results of 
10 estimating the background colors based on the 
locally-executed color threshold processing; 

Fig . 11 shows a show-through removed image with respect 
to the image in Fig. 4 when the size S is set to 2 mm; 

Fig . 12 shows an image based on a binary edge difference 
15 with respect to the image in Fig. 11 when the size S is set 
to 2 mm ; 

Fig. 13 shows a result of applying this processing 
to the periphery of the black pixels in Fig. 12; 

Fig. 14 shows a result of the processing performed 
20 on the image in Fig. 4; and 

Fig. 15 shows an image subjected to mirror-image 
conversion displayed in a gray scale by taking a difference 
between the images in Fig. 4 and Fig. 14 in order to show 
the removed component. 



25 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 

As actual cases of utilizing the processing for 
show- through may be as follows. That is, embedding of the 
method to a digital color copier, application software of 
5 scanners and digital cameras, or pre-processing in image 
analysis/recognition system of a color document. In this 
invention, considering such variations of the situations 
and environments, the problem to deal with show-through of 
a color image has been analyzed in following approaches. 

10 1) Using only a front side image without having to use 

information for a rear side image: 

A method for inputting images on both sides and 
subtracting the rear side image from the front side image 
requires high-precision registration technology. In 

15 general, non-linear geometrical transformation is in many 
cases added to an input image due to optical and mechanical 
factors. Therefore, this method is difficult to be used 
in any devices except specific input equipment such as a 
double-side scanner or a book scanner. Accordingly, any 

20 method for using only a front side image without having to 
use information for a rear side image has mainly been sought, 
and then some points that can not be dealt with by only the 
information for a single-side image are complemented by 
information for its rear side image. 

25 
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2) Method based on local operation: 

As a case where the method is installed, it is 
considered that the method is embedded into software on PC 
or WS, or a digital color copier. There is a big difference 
5 between these two in restriction to their working memories 
(the number of lines that can temporarily be processed) . 
In the software on the PC or the WS , whole information for 
an image is allowed to be randomly accessed, while in an 
embedded device , only local information is generally allowed 
10 to be accessed. Therefore, the method based on local 
operation of an image has mainly been sought. 

3) Optimal setting of processing parameters based on 
general operation: 

15 Setting parameters to define operational effects is 

essential in an image processing algorithm/ system . Of 
these parameters, some requires different setting for each 
image . When whole information for an image can be accessed, 
the method for optimally setting parameters based on general 

20 image operation can be installed. 

Based on such an approach, it is possible to remove 
show-through from a color image by using only information 
for a single-side image of paper, which is applied to 
general-purpose equipment without depending on a specific 

25 input device. 



Fig. 1 is a block diagram of modules showing a color 
image processing apparatus according to an embodiment of 
this invention. In this embodiment, a color image input 
from image input equipment 1, such as a scanner or a digital 
5 camera, is stored in the RAM 6. A series of processing 
explained later is executed by reading the program stored 
in ROM 7 by CPU 5. A progress report or result on the 
processing is provided to a user through a display unit 2 
such as a CRT. The user inputs and specifies any parameters 

10 required for the processing through a keyboard 3 as necessary . 
Intermediate data (reduced color image, color transformed 
image, edge intensity image, binary edge image, background 
color image, binary edge difference image, show-through 
removed image) produced during performance of the processing 

15 explained later is stored in the RAM 6. The data is then 
read out and corrected or written from and in the RAM 6 by 
the CPU 5 as necessary. The show-through removed image 
generated as a result of the series of processing is read 
out from the RAM 6, and is output to image printing equipment 

20 (printer) 4. 

Fig. 2 is a flow chart showing a color image processing 
method according to this embodiment . This invention mainly 
corresponds to an image processing section of a digital color 
copier. Before the processing in this invention, 

25 pre-processing such as A/D conversion and density correction 



is executed as usual, and gray-scale processing or the like 
is executed after the processing. 

When a color image is input by a scanner 1 (step SI) , 
the image is first reduced (step S2) . By processing the 
5 image producedby reducing the original image , the processing 
is speeded up, a used space in working memory is reduced, 
and installation is simplified. A color coordinate system 
is then transformed (step S3) . In a RGB space, components 
are not highly independent from one another, accordingly, 

10 precision of edge detection and robustness against noise 
may not be increased. Therefore, the coordinate system is 
transformed to an appropriate one to improve the precision 
of edge detection and the robustness against noise. 

Subsequently, edge intensity is calculated (step S4) . 

15 By detecting the edge intensity, a foreground portion of 
a color document such as a text or a line drawing and a 
backgroundportion with higher brightness (light color) than 
the show-through portion can resultantly be retained. An 
edge is then detected by binarizing the portions (step S5) , 

20 and color threshold processing is executed on a region except 
the edge (step S6) . The show-through portion can be 
corrected without entry or storage of the rear side image. 
The front side and the show-through portion can be 
discriminated from each other by performing the color 

25 threshold processing, and color can specifically be 



estimated. The size of the show-through portion to be 
removed can be explicitly specified according to the size 
of a set window. The processing is performed based on local 
operation, which leads to economy on a used space in the 
working memory and simplification of installation. A 
show-through removed image is synthesized based on the 
background color image estimated through the color threshold 
processing (step S7) . 

The edge intensity of the show-through removed image 
is then calculated (step S8) , and an edge difference image 
based on the original image and the show-through removed 
image is generated (stepS9). The image produced as a result 
of the processing is compared to the original image, so that 
any part where the image is degraded due to inadequate 
processing can be detected through a simple operation. By 
executing again the processing around the part, degradation 
in the image can be prevented, thus improving the result 
of processing. A difference between the respective edge 
intensity is binarized (stepSlO) , the processing scale (the 
size of a window) is reduced, the color threshold processing 
is performed in the same manner as that in step S6 to estimate 
a background color image (step Sll), and a show-through 
removed image is synthesized (step S12) . Regarding the 
color threshold processing for correction using a 
small-sized window, the size of a show-through portion to 



be removed depends on local characteristics (e.g., 
complicity) of the front side image, but is determined 
depending on the size of the window in a complicated portion . 
By reducing the size of the show-through portion to be removed 
5 degradation in the front side image can be prevented. 

Subsequently, a show-through portion is estimated 
(determined) by calculating a difference between the 
show-through removed image and the reduced original image 
(step S13). Lastly, the show-through portion in the 
10 original image with original resolution is removed (step 
S14) , and the image is output to the printer (step S15) . 

The characteristics of the color image processing 
method according to this invention is explained below. 
1) Utilization of color edge intensity: 
15 A show-through portion is input after passing through 

paper, which gives the portion the same effect as a low-pass 
filter that acts on the portion. The characteristics of 
this filter depend on physical characteristics of paper, 
therefore, it is difficult to specify them quantitatively. 
20 However, it is natural to assume that the edge intensity 
of the show-through portion is lower as compared to that 
of the front side image . In order to calculate edge intensity 
with high precision, it is desirable to make use of edge 
detection [A. Cumani, "Edge detection in multispectral 
25 images," Graphical Models and Image Processing, vol. 53, 



no. l,pp. 40-51, 1991.], [H.-C. Lee and D.R. Cok, "Detecting 
boundaries ina vector field, " IEEE Trans . Signal Processing, 
vol. 39, no. 5, pp. 1181 - 1194, 1991.] not based on a method 
for simply summing edge intensity calculated in each 
5 component of RGB, but based on a vector method for 
transforming the color coordinate system to an adequate one 
and using three components combined. Further, by 
performing appropriate threshold processing on the edge 
intensity, th-e edge of show-through portion can be removed. 

10 At the time of selecting thresholds, the edge intensity 
distribution is statistically analyzed and values are 
automatically set, or a plurality of thresholds are 
previously provided and a user sets values according to a 
degree of show-through or paper quality. 

15 2) Estimation of a background color based on 

region-selectable and locally-executed color threshold 
processing : 

It should be considered that a high-frequency 
component, in particular, such as a character or a line of 

20 the show-through portion is removed. It is assumed now that 
a window in which no edge exists is set in an image. The 
interior of this window can be classified into regions from 
viewpoints of: (a) whether there is show-through in the 
interior, (b) whether the interior includes a region where 

25 the front side image consists of a single color, (c) whether 



it includes a region where the front side image has color 
gradation, and (d) whether it includes a region formed with 
a plurality of different colors because an important edge 
of the front side image has been removed by thresholding. 
5 When viewed from a local point, the show-through 

portion has generally lower brightness as compared to that 
of the front side image. Therefore, the interior of the 
window is classified into two colors, and the interior of 
the window is replaced with the color with higher brightness 

10 (lighter) . Further, although color clustering generally 
requires repetitive calculation, calculation efficiency is 
enhanced here by using a high-speed color threshold 
processing algorithm [S.C. Pel and CM. Cheng, "Color image 
processing by using binary quaternion-moment-preserving 

15 threshold processing technique," IEEE Trans. Image 
Processing, vol. 8, no. 5, pp. 614 - 628, 1999.] based on 
a quaternion representation of an image and a moment 
preserving principle. Further, in order to determine 
"scale" of the processing, the maximum size S of the window 

20 is specified. Through this operation, when a region within 
the window has the characteristic of (b) , show-through is 
removed. However, when this operation is executed on a 
region having the characteristic of (c) or (d) , the image 
is degraded or distorted if the maximum size S is too large. 

25 It will be considered how to deal with the problem in (4) 



explained below. 

3) Image synthesis: 

A show-through removed image is synthesized by using 
the original image for the periphery of the edge and the 
5 background color estimated in the characteristic (2) for 
any portion other than the edge. 

4) Correction to show-through removed image based on 
multi-scale analysis of an edge difference: 

As is clear from the operation of (1 ) , the edge included 

10 in the show-through removed image is supposed to be a partial 
cluster of edges of the original image . Viewing each pixel , 
the edge intensity in the show-through removed image can 
not possibly be higher than that in the original image. If 
there is any edge, that does not exist in the original image, 

15 in the show-through removed image, this is a side effect 
produced due to the fact that the processing scale (maximum 
size S of the window) is too large in the operation (2) . 
In order to correct this effect, estimation of a background 
color is performed again in a smaller scale around the dummy 

20 edge, so that the show-through removed image is corrected. 
It is desirable to determine appropriately the processing 
scale adequate to each image, but this is quite difficult 
in an actual case. Accordingly, the appropriate scales are 
determined one after another while the result of the 

25 processing is compared to the original image based on the 



edge information using a coarse-to-fine strategy. 

One characteristic of this invention is to perform 
local color threshold processing using a window of size S 
(which is explained later with reference to a detailed flow 
5 chart) . This parameter S means the maximum size of a 
show-through portion to be removed, and a show-through 
portion of the size within a rectangle of approx . SxS is 
detected. Further, the value of S is different between a 
portion of a complicated image and a portion of a simple 

10 image, so that the value is required to be adjusted 
appropriately. Accordingly, in this invention, the value 
of S is controlled by comparing the edge intensity of the 
original image with that of the show- through removed image, 
and the processing for locally-executed color threshold 

15 processing is incorporated. 

Further, another characteristic of this invention is 
to speed up the processing by using a reduced image (e.g. , 
400 dpi is reduced to 100 dpi by converting an average of 
colors for each 4x4 pixels to a color for one pixel) . For 

20 example, when show-through of the size of about 2 ram is to 
be removed, in the image reduced to 100 dpi, S is 8 pixels. 

Fig. 3 is a flow chart showing the color image 
processing method according to an embodiment of this 
invention in detail. Each segment of the algorithm is 

25 explained in detail with reference to Fig. 3. 



At step S31, the resolution of a color image read by 
a scanner or the like is reduced, and a color image I (RGB) 
is output. 

Color coordinate transformation: 

Coordinates of the color image I are transformed to 
another color coordinate system to be a transformed color 
image Iq (step S32) . Components in the RGB space are not 
highly independent from one another, therefore, precision 
of edge detection may not be improved. Accordingly, the 
color coordinate is transformed to, for example, YCbCr in 
eguation (1) or a pseudo KL color coordinate system in 
equation (2) . 




Edge detection: 

Edge detection is executed on the transformed color 
image Iq to generate an edge intensity image Eq (step S33) . 
As araethod for edge detection from a color image , the simplest 
method is to seek a square sum root of respective edge 
intensity calculated by a Sobel operator or the like applied 
to each component. There is another method for edge 



detection based on a vector method as explained in the cited 
document [Edge detection in multispectral images (A. 
Cumani) ] , [Detecting boundaries in a vector field (H.-C. 
Lee and D.R. Cok) ] , and it has been known that robustness 
against noise in the method is more excellent than a simple 
method. In the edge detection method based on a vector method 
a square sum root is not simply obtained in combination of 
the edge intensity calculated in components but correlation 
between the components is considered. More specifically, 
assuming that u (x, y) , v (x, y) , and w (x, y) are three 
components of color and these three components are defined 
by p (equation (3) ) , t (equation (4) ) , and q (equation (5) ) , 
so that the edge intensity at a pixel (x, y) is obtained 
by equation (6) . 

(duY fdvY fdwY 

^{p + q + ylip + qT-^ipq-t')) ... (6) 

Fig. 4 shows an example of a color image with 
show-through, and Fig. 5 shows an edge intensity image Eq 



generated by a vector Sobel operator applied to the image 
in Fig. 4. In Fig. 5, pixels with high intensity are 
represented in black. 

Threshold processing and expansion processing of edge 
5 intensity: 

The threshold processing and expansion processing of 
edge intensity are executed on the edge intensity image Eq 
to generate a binary edge image Fo (step S34) . Thresholds 
in the threshold processing for the edge intensity are 

10 automatically set by statistically analyzing intensity 
distribution, or a plurality of thresholds are previously 
provided and any of them is set by a user according to a 
degree of show-through or paper quality. 

Fig. 6 shows a result of binarizing the color edge 

15 intensity in Fig. 5. In the binary edge image Fq, a pixel 
whose edge intensity is higher than the threshold is 
represented as ON, and any pixel other than the pixel is 
represented as OFF. Further, in the binary edge image Fo, 
the expansion processing is subjected to the pixel of ON 

20 as required. 

Estimation of a background color based on locally-executed 
color threshold processing: 

The locally-executed color threshold processing for 
the transformed color image lo is then performed on a region 

25 where the binary edge image Fo is OFF to generate a background 



color image Bo (step S35) . Thatis, alocal background color 
for a region where pixels are OFF on the binary edge image 
Fo is estimated. Further, in the original image, by 
replacing the OFF-pixels on the binary edge image Fq with 
5 the background color image Bq, a show-through removed image 
Jo is generated (step S36) . The processing for estimating 
a background color based on this local color threshold is 
explained in more detail. 

Fig. 7 and Fig. 8 are flow charts showing in detail 
10 the locally-executed color threshold processing using a 
window of size S. The processing for estimation of a 
background color is explained with reference to Fig. 7 and 
Fig. 8. 

This locally-executed color threshold processing is 
15 performed, assuming parameters such as the size of the 
window: s, the width of the image: w, and the height of the 
image: h, by inputting a color original image I and a binary 
edge image F to generate a background color image J as a 
final output. 

20 At first, the parameters and the images I and F are 

input (step S51) . A count value hO of height of the image 
is then set to 0 (step S52) . At step S53, it is determined 
whether hO satisfies hO < h (step S53) . When the value hO 
is greater than h, the processing proceeds to step S72 

25 explained later. When the value hO is smaller than h, a 



count value wO of a width of the image is set to 0 at step 
S54. At step S55, the widths w and wO of the images are 
compared to each other. When the value wO is greater than 
w, the value hO is incremented by one (step S63) , and the 
5 processing returns to step S53 . When wO is smaller than 
w, the processing proceeds to step S56, and it is detected 
whether the binary edge image F[hO] [wO] of the value hO and 
the value wO is 1, that is, whether the image has an edge. 
If any edge exists, the color original image I [hO] [wO] is 

10 determined as a background color image J [hO] [wO] (stepS57), 
the value wO is incremented by one (step S58) , and the 
processing returns to step S55 . If it is determined at step 
S56 that no edge exists , a run R of length t=s in the horizontal 
direction is extracted from the color original image I at 

15 thepoint (hO , wO) as its startingpoint (step S59) . However, 
when there is a pixel as F [hO] [j] = 1 (wO<j<wO + s) , a run 
R of length t= jO-wO is extracted assuming that jO is such 
a pixel and j is the minimum. 

The run R is then classified into two colors and the 

20 color with higher brightness is defined as b (step S60) , 
and J [hO][j] is defined as b when j=wO, wO + 1, wO+t-1 
(step S61) . Subsequently, the value wO is incremented by 
t (stepS62) , andtheprocessingreturns to step S55 . Finally 
the operation is repeated during the period when wO<w and 

25 hO<h are satisfied, and from then on, the processing 



continues to step S72 . 

A run R in the vertical direction is extracted in the 
same manner as the method for extracting the runs in the 
horizontal direction. Actually, at steps S72 to S76, the 
5 transposed operations in the horizontal direction and the 
vertical direction at steps S52 to S56 are executed. At 
step S76, if it is determined that no edge exists, a run 
R of length t=s in the vertical direction is extracted from 
the background color image J based on (hO, wO) as a starting 

10 point (step S79) . However, when there is a pixel as F [i] 
[wO] = 1 (hO<i<hO + s) , a run R of length t=iO-hO is extracted 
assuming that 10 is such a pixel and i is the minimum. 

The run R is then classified into two colors and the 
color with higher brightness is defined as b (step S80) , 

15 and J [i] [wO] is defined as b when i=hO , hO + 1 , • • • , hO+t-1 
(step S81) . Subsequently, the value hO is incremented by 
t (step S82) , and the processing returns to step S75 . Finally, 
the operation is repeated during the period when hO<h and 
wO<w are satisfied, and then, the processing is ended. An 

20 example of formation of the runs is explained below. 

Fig. 9A and Fig. 9B show examples of formation of runs 
in order to estimate a background color. Fig. 9A shows runs 
in the horizontal direction, and Fig. 9B shows runs in the 
vertical direction. As shown in Fig. 9A, each run, whose 

25 maximum length is S and which does not include an ON pixel 



of a binary edge image in such a run is successively formed 
in each horizontal line . In each of the runs , pixels forming 
a run is classified into two typical colors. Such "color 
clustering" generally requires repetitive calculation. 
5 However, the high-speed threshold processing algorithm (S.C. 
Pel and CM. Cheng) based on a quaternion representation 
of an image and a moment preserving principle is used. This 
algorithmhas a characteristic in that the two typical colors 
and a classification boundary are obtained as closed form 

10 solutions without performing repetitive calculation. Of 
the two typical colors thus calculated, a lighter color 
(higher brightness) is set to a color of each pixel forming 
the run. As explained above, an image B', which has been 
subjected to estimation of a background color using the 

15 horizontal runs, is obtained. 

In general, the edge in a show-through portion 
(show-through region) is weaker than the edge of a front 
side image, therefore, as a result of threshold processing 
the edge, the show-through portion is included in "a region 

20 with no edge". In the show-through portion, the density 
is lower (dark color) than that of a background color on 
the front side , therefore , "binarization ( two typical color s 
within a region are calculated) " is executed on an SxS region, 
and a region is replaced with a brighter color. Accordingly, 

25 the show-through portion having an Sxs size or less is 



removed. 

It is difficult to arrange an SxS rectangle so that 
the edge is not included in the rectangle when there is an 
edge . Therefore , as explained above , runs in the horizontal 
5 direction in order to estimate a background color are formed 
{Fig. 9A) . The maximum length of the run is S, and the runs 
are formed so that ON pixels of a binary edge image are not 
included in the runs. "Binarization (two typical colors 
within a region are calculated) " is executed on each of the 

10 runs , and the region is replaced with a brighter color . With 
respect to the image generated as a result, runs in the 
vertical direction are formed in the same manner as explained 
above, and the same processing as that in the case of the 
horizontal direction is performed (Fig. 9B) . 

15 Fig. lOA and Fig. lOB show images as results of 

estimating the background colors based on the 
locally-executed color threshold processing. Fig. lOA 
shows the image as a result of setting S to 2 mm in the image 
of Fig. 4 and estimating a background color using the 

20 horizontal runs. Fig. lOB shows the image as a result of 
estimating a background color further using the vertical 
runs for the background color image in Fig. lOA. 

Further, as shown in Fig. 9B, the runs in the vertical 
direction are formed, and the generated image B ' is subj ected 

25 to the processing for estimating a background color. 



Brightness of each pixel in the background color image B 
generated as a result of the processing (which is explained 
as J in the flow chart of Fig. 7 and Fig. 8) becomes higher 
(lighter) than B'. 
5 By subjecting the runs in the horizontal direction 

and the vertical direction to the processing for estimating 
the background color in such a manner, a background color 
image Bo is generated (step S35 in Fig. 3) . Further, in 
the original image, by replacing the pixels in which the 
10 binary edge image Fo is OFF with Bq , a show- through removed 
image Jo is obtained (step S36) . 

Fig. 11 shows the show-through removed image Jo with 
respect to the image in Fig. 4 when the size S is set to 
2 mm. 

15 Correction to a show-through removed image based on 
multi-scale analysis of an edge difference: 

Correction to a show-through removed image based on 
multi-scale edge difference analysis is further explained 
with reference to the flow in Fig. 3. The size S of the 

20 window is reduced (S/2 in this embodiment) (step S37) , and 
it is determined whether the window size S is greater than 
or equal to one pixel (step S38) . If S is greater than one 
pixel, the processing proceeds to step S43 and thereafter. 
When S>1 , a show-through removed image is corrected based 

25 on the multi-scale edge difference analysis, which will be 



explained below. 

An edge in the show-through removed image Jo is detected 
at each pixel (step S39) , and the edge intensity Eq of the 
original image is subtracted from the detected edge intensity 
5 El to generate an edge difference image (step S40) . Inmost 
of the pixels, each difference (residual) becomes a value 
of 0 or less. This edge difference image is binarized to 
generate a binary edge difference image D. A pixel whose 
difference is the threshold or more is indicated as ON, and 

10 the other pixels are indicated as OFF. 

Fig. 12 shows a binary edge difference image D with 
respect to the image in Fig. 11 when the size S is set to 
2 mm. In Fig. 12, as represented by a circular arc section 
in the lower part of Fig . 11, the portion, that is not regarded 

15 as an edge as a result of binarizing the difference between 
the edge intensity Ei in the show-through removed image Jq 
in Fig. 11 and the edge intensity image Eq in Fig. 5, does 
not remain as an edge . Conversely, the portion that remains 
as an edge is determined as ON pixels, and the processing 

20 proceeds to step of estimating a background color explained 
below . 

The processing for estimating a background color is 
executed on the periphery of ON pixels in the binary edge 
difference image D by using the new value of S that has been 
25 reduced to the half at step S37 , and a background color image 



Bi is generated (step S41) . In the show- through removed 
image Jo, the color of ( 2 S + 1 ) x ( 2 S + 1 ) pieces of pixels near 
the ON pixels in the image D is replaced with color in the 
new background color image Bi (step S42) . 
5 Fig. 13 shows a result of applying this processing 

to the periphery of the black pixels in Fig. 12. The 
processing for correction is recursively repeated until the 
length S becomes less than one pixel , or until no pixel remains 
after the edge difference image is thresholded. 

10 When the show-through removed image Jo is finally 

generated, this color coordinate system is reversely 
transformed to be restored to RGB (step S43) , and the 
show-through portion is identified (step S44) . The 
identification of the show-through portion is performed by 

15 calculating a difference between the show-through removed 
image Jo and the reduced original image. In an actual case, 
a dif^ference between three components of a color is 
calculated, the difference is converted to brightness, or 
an Euclidean distance or a distance in a Lab color space 

20 is calculated. The distance is thresholded to detect a 
show- through portion. 

Lastly, the pixels determined as a show-through 
portion in the original image with the original resolution 
are replaced with the color in the show-through removed image 

25 (step S45) . In this replacement, the color of the pixels 



on the show-through removed image obtained from the reduced 
image is allocated to a corresponding region on the original 
image with the original resolution . For example , one pixel 
of a 100 dpi-reduced image corresponds to 4x4 pixels of 
5 the original image. 

Fig. 14 shows a result of the processing performed 
on the image in Fig. 4. Fig. 15 shows an image obtained 
by taking a difference between the images in Fig. 4 and Fig. 
14 displayed in a gray scale in order to show the removed 

10 component. 

Although the color image processing method and 
apparatus according to this invention have been explained, 
as the embodiments of this invention, a computer-readable 
recording medium, where a program for making a computer 

15 execute the color image processing method or for making a 
computer execute the processing of the color image processing 
apparatus is recorded, can also be provided. 

According to this invention, advantageous effects as 
follows are produced. 

20 (1) By detecting edge intensity, a foreground portion of 

a color document such as a text or a line drawing and a 
background portion where brightness is higher (light color) 
than that of a show-through portion can be retained. 
<2) By color threshold processing, the show-through 

25 portion can be corrected without entry or storage of a rear 



side image. Further, the front side and the show-through 
portion can be discriminated, and a color can specifically 
be estimated. The size of a show-through portion to be 
removed can explicitly be specified by the size of a window 
5 to be set, and the processing is performed based on local 
operation, which leads to economy on a used space in the 
working memory and simplification of installation. 

(3) By generating an edge difference image between the 
original image and the show-through removed image, that is, 

10 by comparing tiie original image with the image produced as 
a result of the processing, any portion where the image is 
degraded due to inadequate processing can be detected by 
a simple operation. By executing again the processing on 
the periphery of the portion, it is possible to prevent 

15 degradation in the image and improve the result of the 
processing . 

(4 ) The size of a show- through portion to be removed depends 
on a local characteristic (complexity or the like) of a front 
side image. However, in its complicated portion, by 

20 reducing the size of a show-through portion to be removed 
that is determined based on the size of a window (color 
threshold processing for correction using a small-sized 
window) , degradation in the front side image can be 
prevented . 

25 (5) The color coordinate system of the RGB space, where 



components are not highly independent from one another and 
precision in edge detection and robustness against noise 
' may not increase, is transformed to an appropriate one, so 
that the precision in edge detection and the robustness 
5 against noise are improved. 

(6) A low-resolution image is produced, and the processing 
for removing show-through is subjected to the image, a 
show-through portion is detected by comparing the image with 
its original image, and the detected portion is replaced 
10 with an appropriate color. That is, the image produced by 
reducing the original image is subj ected to processing , which 
makes it possible to speed up the processing, economize on 
a used space in the working memory, and simplify 
installation . 

15 (7) No specific input equipment is required, so that this 

invention can be applied to any general-purpose image input 
equipment . 

The present document incorporates by reference the 
entire contents of Japanese priority documents, 11-354412 
20 filed in Japan on December 14, 1999. 

Although the invention has been described with respect 
to a specific embodiment for a complete and clear disclosure , 
the appended claims are not to be thus limited but are to 
be construed as embodying all modifications and alternative 
25 constructions that may occur to one skilled in the art which 



fairly fall within the basic teaching herein set forth. 
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